Exploiting Object Dynamics for Recognition and Control

This thesis explores a new approach to object recognition for active vision systems that integrates information across multiple observations of an object. The approach exploits the order relationship between successive frames to derive a classifier based on the characteristic motion of local features across visual sweeps. This motion model reveals structural information about the object that can be exploited for recognition. The first contribution of this thesis is a recognition system that extends invariant local features (shape contexts) into the time domain by integration of the motion model. Second, an entropy-based view selection scheme is presented that allows the vision system to “skip ahead” to highly discriminative views. Evaluations on two datasets demonstrate that both the motion model and active view selection yield higher-quality hypotheses about object categories quicker than a baseline system that treats object views as unordered streams of images. Thesis Supervisor: Deb Roy Title: Associate Professor of Media Arts and Sciences, Program in Media Arts and Sciences